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ABSTRACT 

The National Assessment of Educational Progress 
(NAEP) is working to increase the numbers of students with 
disabilities (SD) or limited English proficient (LEP) students who 
are included in the assessment. Information is presented oh the 
changes made to the inclusion criteria, the types of accommodations 
being offered, new procedures implemented in 1996, and ongoing 
research studies. The percentages of SD and LEP students excluded 
have been fairly steady over time, with about 5% excluded due to 
Individualized Education Plans (lEPs) and 1% excluded due to LEP. An 
evaluation conducted in 1994 found that many of these students were, 
in fact, capable of participating in the assessment. In preparation 
for the 1996 assessment, the NAEP field tested the new inclusionary 
criteria and the use of various accommodations and adaptations for 
the mathematics assessment. Field test results were generally 
encouraging, but did indicate that results for SD and LEP students 
may not be comparable to those of other students. A special sample 
design was developed for the 1996 NAEP to help clarify issues raised 
by the inclusion of additional SD and LEP students. Current research 
is examining scaling, reporting, appropriateness, construct validity, 
language complexity, and inclusion issues. (SLD) 
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Increasing the Inclusion of Students with 
Disabilities and Limited Engiish Proficient 
Students in NAEP 



The purpose of this edition of Focus on NAEP \s to 
describe the activities underway in the National As- 
sessment of Educational Progress (NAEP) toward 
increasing the numbers of students with disabili- 
ties (SD) or limited English proficient (LEP) stu- 
dents who are included in the assessment In this 
article, information is provided on the changes made 
to the inclusion criteria, the types of accommoda- 
tions now being offered, new procedures imple- 
mented in 1996, and ongoing research studies on 
inclusion issues. 



NAEP is in the process of examining and implement- 
ing new procedures to maximize the repre- 
sentativeness of students included in the assessment. 
The focus is on efforts to enhance overall participa- 
tion by including more SD and LEP students. Inclu- 
sion is receiving increased attention at the federal, 
state, and local levels, with much activity occurring in 
recent years. Changes have been made in the criteria 
for making selection decisions to include more stu- 
dents in the assessment and to offer various accom- 
modations to students with special needs. Currently, 
NAEP is looking carefully into a variety of issues 
having to do with inclusion of both the SD and LEP 
student populations. 



NS 



NAEP’s main purpose is to provide key indicators of 
what the nation’s students know and can do. This 
concept means that the NAEP results should repre- 
sent students in the nation. This is especially im- 
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portant because NAEP uses a sampling approach in 
which all students may be included, although, in actu- 
ality, some students with disabilities and limited Eng- 
lish proficiency do not participate in the assessment. 
Recent educational trends, reflected in the authoriza- 
tions of Goals 2000: Educate America Act and Im- 
proving America’s Schools Act (lASA), and the 
proposed reauthorization of the Individuals with Dis- 
abilities Education Act (IDEA), have called for as- 
sessments that are meaningful, challenging, and 
appropriate for all students. This call has led NAEP 
to look closely at the procedures for assessing larger 
numbers of SD and LEP students. NAEP is commit- 
ted to increasing participation of as many of these 
students as possible in die assessment. Among the 
many benefits of this approach are improved meas- 
urement of overall student achievement, enhance- 
ment of the representativeness and generalizability of 
NAEP results, and greater fairness and equity. 

Among the challenges for NAEP are upholding its 
high degree of validity, maintaining its reliability, and 
presenting the ability to analyze and report trends in 
the face of changes made to procedures and the sam- 
pled population of respondents. 

NAEP’s Approach Prior to 1995 

In the past, students with disabilities and limited Eng- 
lish proficiency were often excluded from NAEP for 
several reasons — state and local policies had been de- 
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Table 1. — NAEP inclusion rates for lEP and LEP students, by grade: 1992—1994 



Grade 4 
1992 
1994 


Percent Students Percent lEP 

Identified lEP Students 

Assessed 


Percent Students Percent LEP 

Identified LEP Students 

Assessed 


9 33 

12 50 


4 25 

6 50 


Grade 8 
1992 
1994 


9 44 

13- 38 


3 33 

4 50 


Grade 12 
1992 
1994 


5 20 

11 36 


2 50 

3 66 

\ 



signed to identify SD and LEP students and exclude 
them from testing based on certain criteria; school staff 
may have believed these students were unable to par- 
ticipate meaningfully; or, no test accommodations or 
adaptations were available that met the specific needs 
or requirements of the Individualized Education Plans 
(lEPs) required by law for students with certain disabili- 
ties. In order to standardize NAEP procedures, NAEP 
had previously developed policy guidelines for includ- 
ing students with disabilities or limited English profi- 
ciency, but some were excluded, particularly those with 
profound disabilities or students who would have re- 
quired accommodations to the testing procedures. 

In previous years, about half or more of the students 
identified as lEP or LEP were excluded from the 
NAEP assessments (see table 1). The percentages of 
students excluded, because in the judgment of school 
staff they could not participate meaningfully, have been 
fairly steady over time, with about five percent ex- 
cluded due to lEPs and one percent excluded due to 
LEP. While the percentage of students excluded has re- 
mained relatively small in relation to the total popula- 
tion, these students make up a relatively large portion 
of the special education and LEP populations. Deci- 
sions to exclude students from the assessment were 
based on specific criteria used with NAEP (discussed 
further in die next section). Exclusion was further im- 
pacted by a lack of available accommodations and adap- 
tations (such as extended testing time or assessment 
instruments in other languages). 



An evaluation of NAEl^ conducted by the National 
Academy of Education in 1994 found that many of 
the SD and LEP students who had been excluded from 
NAEP were, in fact, capable of participating in the as- 
sessment. This was particularly true if certain types of 
adaptations and accommodations could be offered for 
the assessment. It also recommended that NAEP de- 
velop better criteria to promote inclusion, rather than 
exclusion, of students. During this same time period, 
various offices in the Education Department as well as 
other groups also were involved in examining the issues 
related to including more students in assessments. 
These groups provided additional valuable input to 
NAEP on ways to improve the inclusion and accom- 
modations process. In response to these concerns, 
NAEP explored ways to increase the inclusion rates 
even further. To meet these objectives, plans were 
made to try out new approaches toward inclusiveness 
in preparation for the 1996 assessment. 

Changes in Procedures in 
Preparation for the 1996 NAEP 

The exclusion or inclusion criteria used in NAEP have 
changed somewhat over time. Prior to 1995, the proce- 
dures used by NAEP to determine who can participate 
in the assessment were based on criteria for excluding 
students. Beginning with the 1995 NAEP field test, the 
criteria were revised to be more inclusionary. Other 
changes were made to make a better link between sub- 
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Table 2. — Old and new exclusion or inclusion rules for NAEP 





Old (1990-1994) 


New (1995-1996) 




Exclude if: 


Include if: 


Students 

with 

Disabilities 


Student is mainstreamed less than 50 
percent of the time in academic subjects 
and is judged incapable of participating 
meaningfully in the assessment, OR 

lEP team or equivalent group determine 
that the student is incapable of participating 
meaningfully in the assessment. 


Student has an lEP, unless the lEP team or 
equivalent group determine that the student 
cannot participate, or if the student’s cognitive 
functioning is so severely impaired that he or 
she cannot participate, even with accommoda- 
tions. 


Students 

with 

Limited 

English 

Proficiency 


Stydent is native speaker of a language 
other than English, AND 

Enrolled in an English-speaking school 
(not including bilingual education program) 
for less than two years, AND 

Judged to be incapable of taking part in 
the assessment. 


Student has received academic instruction 
primarily in English for at least three years, OR 
Student has received academic instruction in 
English for less than three years, if school 
staff determine that the student is capable of 
participating in the assessment in English, OR 
Student, whose native language is Spanish, 
has received academic instruction in English for 
less than three years, if school staff determine 
that the student is capable of participating in the 
assessment in Spanish (if available). 



ject areas of instruction and the assessment and in re- 
sponse to concerns about the language used in the old 
criteria. The old and new criteria are presented in table 
2 . 

In all years, schools were instructed that SD and LEP 
students should be assessed if, in the judgment of 
school staff, they were capable of taking the assess- 
ment, and that, in cases of doubt, the school should err 
on the side of inclusion. 

In preparation for the 1996 assessment, NAEP field 
tested in 1995 the new inclusionary criteria for partici- 
pation and the use of various accommodations and ad- 
aptations for the mathematics assessment. The field 
test also included science, but no accommodations or 
adaptations were tested for this subject. For mathemat- 
ics, accommodations were made available for students 
with disabilities if they were part of the student’s nor- 
mal testing procedure, as specified in the student’s lEP. 

The accommodations for students with disabilities in- 
cluded 

• provision of large-print booklets and large- face 
calculators; 

• provision of Braille booklets and talking calculators; 



• accommodations in administration procedures, 

(e.g, unlimited testing time, individual or small- 
group administrations, allowing a facilitator to read 
directions, allowing students to give answers orally, 
allowing students to give answers using a special 
mechanical apparatus). 

Accommodation and adaptation strategies for limited 
English proficient students (provided for mathematics 
only) included the availability of 

• Spanish-English bilingual assessment booklets, 
with items in different languages presented on 
facing pages; 

• Spanish-only assessment booklets. 

In general, the 1995 field test results were encouraging. 
Some SD and LEP students who would not have par- 
ticipated under previous assessment conditions were 
able to participate in the field test. The results of the 
field test showed that the new procedures could be im- 
plemented successfully in the 1996 national assessment. 
However, a preliminary analysis of the test items and 
student performance indicated that, for both the SD 
and LEP samples of students included and assessed un- 
der nonstandard conditions (i.e., with accommodations 
or adaptations) in the field test, the results may not be 
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Table 3. — Sample design for 1996 NAEP mathematics assessment 



Sample and Purpose 


Inclusion Criteria 


Accommodations 


1 - Trends to past years 


Old (1990-1994) 


No 


2 - Evaluate new criteria 


New (1995-1996) 


No 


3 - Evaluate new criteria and accommodations 


New (1995-1996) 


Yes 



comparable to those from other students. Further 
study of the statistical and measurement issues was indi- 
cated for 1 996. 

Design, Inclusion Criteria, and 
Accommodations for the 1996 
NAEP 

In preparing for the 1996 assessment, NAEP staff 
were concerned about several conflicting issues related 
to possible changes that were being planned, including: 
(1) the measurement of trends, (2) the comparability of 
results from students who were accommodated with 
those who were not, and (3) the impact of changes in 
the inclusion criteria and provision of accommodations. 

To address these issues, a special sample design was de- 
veloped to examine carefully the effects of these 
changes. The 1996 NAEP was designed to allow meas- 
urement of trends from past years, necessitating follow- 
ing the same procedures as in the past, while at the 
same time incorporating new criteria for inclusion of 
students with disabilities and LEP students and offer- 
ing accommodations and adaptations for students 
needing them. The design allowed research into the im- 
pact of changes in the inclusion criteria and the use of 
accommodations and adaptations. To address these 
needs, the 1 996 national-level NAEP in mathematics 
actually consisted of three separate samples of about 
3,500 students each at grades 4, 8, and 12, as shown in 
table 3. 

In Sample 1, to measure trends to past years, SD and 
LEP students were included using the same criteria as 
in 1990-1994 and no accommodations were offered. 

In Sample 2, to evaluate the effect of using new inclu- 
sion criteria only, SD and LEP students were included 
using the new criteria, but no accommodations were of- 
fered. In Sample 3, to evaluate the impact of using new 
criteria and offering accommodations, SD and LEP stu- 
dents were included using the new criteria and accom- 
modations were provided. 



By comparing average results and inclusion rates 
among the three subsamples, the impact of the new 
inclusion criteria and of providing accommodations 
will be revealed. At the same time, results from stu- 
dents in the first sample and from similar students in 
the other two samples can be compared to results from 
previous years because testing procedures will not have 
changed for these students. In the 1996 science assess- 
ment, only the second and third subsamples were used 
because there were no previous assessments in science 
to which 1996 results will need to be compared. 

Initial results pertaining to students with disabilities or 
limited English proficiency are planned to be reported 
in the 1996 NAEP Report Cards in mathematics and 
science in early 1997. One chapter in each report will 
describe procedures to increase inclusion and show the 
percentages of students in the subsamples who cor- 
recdy answered selected assessment items. A later re- 
port will be devoted entirely to findings on the impact 
of including and accommodating SD and LEP students 
in the assessment. 

Ongoing Research Studies on 
Inclusion Issues 

Because this is a pioneering effort by NAEP, and ex- 
amination of the topic is difficult because of its many 
complexities, additional research is required. A number 
of studies are currendy underway in NAEP to investi- 
gate further issues related to inclusion of students with 
disabilities or limited English proficiency. The research 
includes studies of: 

• Scaling issues — an investigation to determine if there 
is a lack of fit between responses of students given 
under nonstandard assessment conditions with 
those given under standard conditions, and if so, 
what may account for it; 

• Reporting issues — an examination of the implications 
of reporting NAEP in alternate ways (e.g., 
percent-correct metric) for SD and LEP students 
when there is a lack of fit on the NAEP scale; 
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• Appropriateness of the inclusion criteria for SD and LEP 
students — for students with disabilities, a 
determination of the proper role of the lEP team; 
for LEP students, a closer look at whether the 
threshold for English-language study should be 
two or three years; 

• Construct validity of the assessment for SD and LEP 
students — comparisons of the response patterns of 
students assessed under nonstandard conditions 
(e.g., with accommodations) with those of students 
assessed under standard conditions, and how they 
may differ; determination if NAEP is accurately 
measuring what accommodated students know and 
can do; 

• language complexity issues — ^to examine whether 
native language proficiency of LEP students affects 
performance on NAEP and what language 
adaptations could be implemented into the 
assessment for these students; another study will 
examine if training scorers to recognize typical 
syntax, spelling, and other errors made by LEP 
students can improve the scoring of their 
responses to extended constructed-response items; 
and 

• Inclusion procedures — to investigate and determine if 
there still are students being excluded who could 
be included in the assessment. 

It is hoped that this collection of studies will help 
NAEP make further progress toward the goal of full in- 
clusion, while preserving the overall validity of the as- 
sessment. Research studies like these are essential in 
order to ensure that the reporting of what students 
know and can do, as well as gauging their academic pro- 
gress over the years, is done in a reliable, valid, and 
meaningful way. As the various issues are examined 
and new procedures implemented, NAEP must main- 
tain its commitment to high technical standards, the 
continuity of its trend data, and a balance of resources 
for the program. 

In conclusion, the changes incorporated into the 1996 
NAEP that further the goal of maximum inclusion of 
students with disabilities or limited English proficiency 
wiU result not only in an improved national assessment 
program, but wiU also benefit states, school districts, 
and other entities that conduct large-scale assessments. 



Many educators at these levels look to NAEP as a 
model for the best practice in assessment. Thus, 

NAEP needs to proceed in a thoughtful and thorough 
manner in its implementation of a more inclusive as- 
sessment. 

Notes 

1 . The Trial State Assessment Prospects and Realities, The 
Third Report of the National Academy of Education 
Panel on the Evaluation of the NAEP 1992 Trial State 
Assessment, (National Academy of Education, Stan- 
ford University: 1993). 

2. (a) Making Decisions about the Inclusion of Students mth 
Disabilities in Large-Scale Assessments — A Report on a 
Working Conference to Develop Guidelines on Inclusion 
and Accommodations, (National Center on Educa- 
tional Outcomes, College of Education, Univer- 
sity of Minnesota: 1994). 

(b) Recommendations for Making Decisions about the Par- 
ticipation of Students with Disabilities in Statewide Assess- 
ment Programs — A Report on a Working Conference to 
Develop Guidelines for Statewide Assessments and Stu- 
dents with Disabilities, (National Center on Educa- 
tional Outcomes, College of Education, 

University of Minnesota: 1994). 

(c) Proceedings of the Conference on Inclusion Guidelines 
and Accommodations for Limited English Prof dent Stu- 
dents in the National Assessment of Educational Pro- 
gress, (National Center on Education Statistics, 
OERI, U.S. Dept, of Education: 1996). 

The Focus on NAEP series briefly summarizes informa- 
tion about the ongoing development and implementa- 
tion of the National Assessment of Educational 
Progress. The series is a product of the National Cen- 
ter for Education Statistics. Gary W. Phillips serves 
as the Associate Commissioner for Education Assess- 
ment. 

This issue was written by fohn F. Olson of the Educa- 
tion Statistics Services Institute (ESSI) 2 svdi Arnold A. 
Goldstein of the National Center for Education Statis- 
tics. Readers are invited to comment by contacting the 
first author at NCES, Education Assessment Division, 
555 New Jersey Ave. NW, Washington, DC 20208— 
5653. To order NAEP publications, call Bob Clemons 
at (202) 219-1690. 
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